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Abstract.  We  describe  our  progress  on  instrumenting  a  Navy  software  simulator  for  use  in  the 
context  of  intelligent  agent  research.  The  Tactical  Action  Officer  (TAO)  Sandbox,  developed  at 
the  University  of  Southern  California,  is  used  by  officers  to  train  for  specific  Navy  missions.  NRL 
and  Knexus  Research  Corporation  have  integrated  this  simulator  with  intelligent  agents  using  the 
Lightweight  Integration  and  Evaluation  Testbed  (LIET),  thus  permitting  the  agent  to  play  the  role 
of  a  trainee.  This  will  permit  us  to  use  the  TAO  Sandbox  in  our  artificial  intelligence  research, 
where  we  are  currently  focusing  on  algorithms  for  continuous  planning  that  can  dynamically 
reason  about  what  goal  should  be  pursued  at  any  time  during  a  mission.  This  paper  briefly 
descibes  our  motivation  for  this  integration,  project  status  involving  this  simulator,  and  future 
goals. 

1.  Motivation 

One  of  the  core  topics  within  the  artificial  intelligence  (AI)  research  community  is  planning,  which 
concerns  the  task  of  synthesizing  a  plan  to  accomplish  a  goal  from  an  initial  state  using  a  set  of  actions 
(Russell  &  Norvig,  2003;  Ghallab  et  al.,  2004).  In  this  context,  a  state  space  S  represents  the  entire  set  of 
possible  states,  an  initial  state  s,  e  S  is  a  point  in  this  space,  a  goal  G  c  S  is  a  set  of  one  or  more  points  in 
this  space,  and  an  action  aeA  such  that  a:  S—>S  is  a  mapping  among  states  (i.e.,  an  action  permits 
movements  among  states).  Thus,  a  planner  p(G,  sit  A)  identifies  a  set  of  actions  aeA  that  can  potentially 
be  executed  so  as  to  traverse  from  s{  to  some  state  in  geG. 

Planning  is  an  old  topic  dating  to  the  earliest  days  of  AI,  and  hundreds  (if  not  thousands)  of  papers  have 
been  devoted  to  it,  ranging  from  theoretical  to  applied,  automated  to  interactive,  and  involving  a  large 
variety  of  approaches  and  assumptions.  Also,  annual  international  conferences  are  now  devoted  to  AI 
planning.  Practical  motivations  for  planning  abound,  from  industrial  robotics  tasks  to  Navy  needs  (e.g., 
controlling  automated  vehicles,  assisting  with  assessing  Course-of -Action  plans,  and  providing  intelligent 
agents  -  neutrals,  allies,  or  adversaries  -  in  training  simulations).  However,  although  a  few  exceptions 
exist  (e.g.,  Cox,  2007),  comparatively  little  research  has  been  devoted  to  developing  methods  that  can 
dynamically/automatically  generate  and  manage  their  own  goals,  which  is  a  desired  capability  of 
intelligent  agents  for  continuous  planning  tasks  (e.g.,  such  as  those  involving  many  Navy  missions). 

Instrumented  simulators  are  valuable  for  AI  research  because  they  can  provide  a  source  for  data  and 
evaluation  metrics  when  integrated  with  intelligent  agents  (Molineaux  &  Aha,  2005).  We  seek  a  simulator 
that  can  serve  as  an  environment  (i.e.,  for  accepting  actions  as  input,  executing  them,  and  providing  state 
information  as  output)  for  conducting  continuous  planning  research  on  dynamic  goal  generation  and 
management.  Furthermore,  we  prefer  to  use  Navy-relevant  simulators  in  our  research.  In  the  following 
sections,  we  describe  such  a  simulator  (the  TAO  Sandbox)  and  its  intended  use  in  this  context. 

2.  TAO  Sandbox 

The  TAO  Sandbox  (TAOS)  is  an  interactive  Navy  training  simulator  for  Tactical  Action  Officers  (TAOs) 
that  is  being  developed  by  Allen  Munro  and  Quentin  Pizzini  at  USC’s  Center  for  Cognitive  Technology. 
This  system  was  designed  with  guidance  from  the  Surface  Warfare  Officer’s  School  (SWOS)  in  Newport, 
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Figure  1:  Snapshot  of  the  TAO  Sandbox  user  interface,  which  can  be  used  by  instructors  to  create 
simulated  ASW  scenarios  involving  the  deployment  and  control  of  helicopters,  sonobuoys,  ships,  and 
sonar  systems.  Trainee  solutions  provided  are  recorded  for  later  review  by  instructors. 

RI,  and  has  been  used  in  the  SWOS  Department  Head  Course  since  2008.  Trainees,  working  in  teams, 
interact  with  the  TAOS  to  solve  conceptual  problems  in  simulated  anti-submarine  scenarios  (that  were 
previously  authored  by  instructors),  during  which  they  address  decision-making  situations  (e.g.,  requiring 
ship  coordination  in  a  hostile  environment)  similar  to  ones  that  they  may  need  to  resolve  later  in 
operational  situations.  Trainee  interactions  can  be  recorded  and  replayed  to  assess  the  students’ 
performance.  The  TAOS  can  (1)  provide  clear  explanations  of  ASW  issues  and  tactics  to  trainees,  (2) 
supply  more  (and  more  varied)  ASW  problems  to  instructors,  and  (3)  allow  users  to  create  new  scenarios 
that  may  require  new  tactics.  Figure  1  highlights  aspects  of  the  TAOS  user  interface  and  system. 

A  TAOS  scenario  includes  a  map  containing  a  human-operator  controlled  group  of  ships,  planes,  and 
helicopters,  along  with  Al-controlled  enemy  submarines  and  miscellaneous  other  ships  that  may  be 
hostile.  The  operator’s  task  is  to  use  their  units  to  accomplish  certain  mission  objectives  such  as: 

•  Locate  and  engage  enemy  submarines 

•  Identify  all  ships  in  an  area  to  find  a  target  ship 

•  Escort  a  Mission  Essential  Unit  (MEU)  to  a  destination  (e.g.,  a  naval  base) 

These  scenarios  can  be  simulated  to  take  place  in  any  part  of  the  world  by  changing  or  adding  a  map  of 
the  desired  area.  USC  is  creating  more  vehicle  actions  and  adding  more  types  of  units,  which  further 
increases  the  system’s  real-world  applicability  and  allows  for  more  in-depth  experiments. 

The  Navy  Center  for  Applied  Research  in  AI  (NCARAI)  at  the  Naval  Research  Laboratory  is  currently 
working  with  this  simulator  as  a  test-bed  for  current  and  future  research.  NCARAI  seeks  Navy  simulators, 
such  as  the  TAOS,  for  conducting  AI  research.  TAOS  is  an  excellent  simulator  to  work  with,  due  to  its 
real-world  application,  use  of  limited  knowledge  (e.g.,  submarine  locations,  ship  identities),  and  software 
system  stability. 


3.  Current  Research 

3.1  The  TAOS  Interface 

To  date,  most  of  NCARAI’s  work  with  the  TAOS  has  concerned  creating  a  software  integration  interface 
for  the  system.  The  simulator  was  originally  designed  with  a  human  operator  in  mind  with  minimal 
support  for  automated  AI.  Even  the  computer  controlled  units  in  TAOS  do  not  have  much  intelligence; 
they  tend  to  travel  on  preset  paths  until  they  come  within  range  of  a  human-controlled  ship,  where  upon 
they  may  fire  a  torpedo.  Given  this,  we  created  an  interface  for  an  intelligent  software  agent  to  control  the 
operator’s  vehicles,  and  to  obtain  observations  about  the  environment’s  state. 

We  designed  this  TAOS  interface  so  that  most  necessary  actions  available  in  the  system  are  now  mapped 
to  different  methods  for  a  controlling  agent.  This  allows  for  incremental  development  with  the  TAOS  and 
eases  maintenance  by  being  able  to  pinpoint  computation  errors.  The  current  interface  can  perform  all  of 
the  required  actions  in  TAOS  version  1.11.  However,  our  interface  is  not  entirely  compatible  with  newer 
TAOS  versions,  which  have  added  features  (e.g.,  new  units  such  as  air  bases,  missile  launchers,  and 
surface  to  air  missiles)  and  changed  some  functionality. 

3.2  TAOS  LIET  Integration 

We  integrated  the  TAOS  with  LIET  (Molineaux  et  al.,  2009),  which  is  a  free  tool  developed  by  Knexus 
that  can  be  used  to  integrate  simulation  environments  (such  as  the  TAOS)  with  intelligent  agents.  LIET 
also  provides  facilities  to  perform  experimentation  and  performance  analysis.  We  used  a  modified 
version  of  the  SHOP2  Hierarchical  Task  Network  Planner  (Nau  et  al.,  2003)  and  a  custom-built  TAO 
domain  as  the  initial  agent.  Our  modification  to  SHOP2  enabled  temporal  planning,  which  is  essential  for 
real-time  environments  like  the  one  supported  by  TAOS  simulator. 

The  TAOS-LIET  integration  has  been  successful.  LIET  can  start  up  the  TAOS,  read  state  information 
from  it,  and  convert  this  information  into  correct  LISP  syntax  for  our  planning  agent.  LIET  can  also 
convert  the  plan  created  by  this  agent  into  actions  that  the  TAOS  simulator  can  execute. 

3.3  Example  Scenario  for  Dynamic  Goal  Reasoning  and  Management 

TAOS  offers  a  large  playground  for  Dynamic  Goal  Reasoning  and  Management  due  primarily  to  the 
open-endedness  of  the  goals  in  the  system  along  with  the  ease  in  which  one  can  create  a  custom  scenario. 
This  means  that  most  reasonable  scenarios  are  only  limited  by  what  objectives  the  researchers  conceive. 
One  simple  scenario  that  requires  dynamic  goal  reasoning  and  management  is  the  task  of  identifying  a 
ship  that  changes  course  to  avoid  being  identified.  In  the  simulation  to  identify  a  ship  the  operator  needs 
to  send  a  plane  or  a  helicopter  to  make  visual  contact  with  an  unidentified  ship.  In  simulation  terms  this 
means  the  aerial  unit  needs  to  be  within  the  visual  range  of  the  ship  as  defined  by  the  current  scenario. 
When  planning  for  identifying  a  ship  there  is  no  way  of  knowing  what  the  ships  course  is  going  to  be  in 
the  future.  The  only  information  available  to  the  planner  is  the  current  speed  and  bearing  of  the  ship.  It  is 
therefore  possible  for  a  ship  to  make  an  erratic  course  change  causing  the  aerial  vehicle  not  to  make  visual 
contact  along  its  flight  path.  An  example  of  this  scenario  is  shown  in  Ligure  2,  where  a  helicopter  was 
assigned  to  identify  a  ship  that  will  soon  after  the  assignment  make  a  180  degree  turn,  causing  the 
helicopter  to  not  fly  into  its  visual  range.  When  the  plan  ends  or  is  updated  the  expectation  of  the  ship 
being  identified  will  not  be  true  and  dynamic  reasoning  or  re-planning  will  be  required  to  fix  the  situation. 
An  explanation  that  the  ship  changed  course  should  be  generated  along  with  a  plan  to  fix  the 
inconsistency.  Some  other  simple  situations  that  require  similar  dynamic  reasoning  are: 

•  While  searching  for  an  enemy  submarine  one  of  the  operator  controlled  ships  is  destroyed  by  a 
torpedo. 

•  While  identifying  ships  a  hostile  or  target  ship  is  spotted. 

•  A  submarine  sighting  is  reported. 
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Figure  2:  Snapshot  of  a  TAO  Sandbox  scenario  executing  a  plan  to  identify  unidentified  ships  on  the  map. 
This  scenario  has  as  indicated  by  the  yellow  line  a  surprise  course  change  for  the  ship  being  identified  that 
will  cause  an  aerial  unit  to  miss  it  when  it  attempt  to  identify  it. 

4.  Future  Work 

We  integrated  the  TAOS  with  LIET  to  conduct  research  on  dynamic  goal  reasoning  and  management. 

Through  iterative  system  testing,  we  will  increase  the  complexity  of  the  HTN  plans  that  our  agent  can 

process,  and  conduct  a  systematic  evaluation  of  its  components. 

We  also  plan  to  use  the  TAOS  in  future  research  projects,  including  one  focused  on  assessing  the  utility 

of  relational  data  representations  and  another  on  studying  the  utility  of  different  types  of  spatial-temporal 

representations  for  supporting  models  of  computational  analogy. 
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